Internet damage from Hurricane Sandy -- the Internet senses its own failures

Still from Renesys animation

This is a follow up on yesterday's post on the Internet damage done by Hurricane Sandy. That post described data center outages and Paul Baran's 1964 RAND reports spelling out the rationale for and design of a packet switched network. It also includes a link to a terrific interview of a data center CEO who struggled to stay on-line during and after Hurricane Katrina in New Orleans.

The Internet senses its own failures in two ways, automatically and in cooperation with humans.

A network is removed from the global routing table a few seconds after it goes down. Renesys tracks the dynamic state of the Internet by monitoring that table.

This animation shows the percent of networks that are down in small geographic areas hit by the storm. Dark green indicates that at least 99.95% of the networks are up and the dark red indicates that more than 5% are unreachable.

They report that in Manhattan the typical outage rate is around 10% and point out that "silencing ten percent of the networks in the New York area is like taking out an entire country the size of Austria, in terms of impact on the global routing table." That is the bad news. The surprisingly good news is that 90% of the data centers are still up -- running on backup diesel power and caffeine.

On the right, we see another Renesys view showing network outage by state over time. As we see, New York was the hardest hit with around 1,200 networks off line at the peak, but some have come back on-line.

Renesys monitoring is automatic, but people are also monitoring the network with the aid of tools like Twitter. Andy Carvin is known for his use of Twitter and other Internet tools in producing real time news reporting (see the presentation at this location) and the same approach has been used in disaster reporting.


Using Twitter and other sources, Rich Miller of the Data Center Knowledge blog has been reporting on data center outages

J.C.R. Licklider

Internet damage is caused by flooding and power outages and the Internet is also used (by people) to report power outages. Those reports are aggregated and mapped in real time at Web sites run by companies like Con Edison, which serves Manhattan and the Long Island Power Authority. (Note the overlap between the power-outage and Internet outage maps).

Finally, let's note that the human-Internet collaboration on disaster reporting or any other task was anticipated long ago by J. C. R. Licklider who, in the 1960s, wrote of man-computer symbiosis, envisioned the Internet and was instrumental in funding much of the research that led to the Internet and modern personal computers.  (Read two of his highly influential papers here).

The Internet damage caused by hurricane Sandy

As GigaOm points out, many data centers are flooded and are running backup generators for power. There has also been some damage to transatlantic cables.

This is reminiscent of the Internet damage caused by Hurricane Katrina in New Orleans. Only one New Orleans Internet service provider, DirectNIC, remained online during and after the hurricane. Shortly after the hurricane, Doc Searles conducted a 40-minute interview of DirectNIC CEO Sigmund Solares. Solares tells a fascinating story of the effort it took to find deisel fuel for their standby generators and keep the data center running. If you are a data center geek -- even a little bit -- you will enjoy the interview.

It also reminds me of one of the initial motivations for the invention of packet switched networks -- that they would be able to route around damaged equipment and continue to function in case of a disaster. The figure below is from one of Paul Baran's 1964 RAND reports outlining the rationale and design for a distributed packet switching network.

The problem here is that too much equipment was located in a small area. You can understand why -- there are many large businesses in lower Manhattan and a lot of cable landing points near New York City.

-----

I've added a second post on the damage caused by the hurricane. It talks about ways the Internet senses and reports its own state.